Data Curation with Deep Learning [Vision]: Towards Self Driving Data Curation

Past. Data curation – the process of discovering, integrating, and cleaning data – is one of the oldest data management problems. Unfortunately, it is still the most time consuming and least enjoyable work of data scientists. So far, successful data curation stories are mainly ad-hoc solutions that are either domain-specific (for example, ETL rules) or task-specific (for example, entity resolution). Present. In the era of big data, data curation plays the critical role of taking the value of big data to a new level. However, the power of current data curation solutions are not keeping up with the ever changing data ecosystem in terms of volume, velocity, variety and veracity, mainly due to the high human cost, instead of machine cost, needed for providing the ad-hoc solutions mentioned above. Meanwhile, deep learning is making strides in achieving remarkable successes in areas such as image recognition, natural language processing, and speech recognition. This is largely due to its ability of (automatically) understanding data (features) that are neither domain-specific nor task-specific. Future. Data curation solutions need to keep the pace with the fast-changing data ecosystem, where the main hope is to devise domain-agnostic and task-agnostic solutions. To this end, we start a new, five-year research project, called AutoDC, to unleash the potential of deep learning towards self-driving data curation. We will discuss how different deep learning concepts (for example, distributed representations, model pre-training, transfer learning, and neural program synthesis) can be adapted and extended to solve various data curation problems. We will also showcase some low-hanging fruits about the early encounters between deep learning and data curation happening in AutoDC. We believe that the directions pointed out by this work will not only drive AutoDC towards democratizing data curation, but also serve as a cornerstone for researchers and practitioners to move to a new realm of data curation solutions. PVLDB Reference Format: Saravanan Thirumuruganathan, Nan Tang & Mourad Ouzzani. Data Curation with Deep Learning. PVLDB, 11 (5): xxxx-yyyy, 2018. DOI: https://doi.org/TBD